The value of using patient‐reported outcomes for health screening during long‐term follow‐up after paediatric stem cell transplantation for nonmalignant diseases

Abstract Introduction The assessment of using patient‐reported outcomes (PROs) within comprehensive care follow‐up programmes, specifically focused on health screening, remains largely unexplored. PROs were implemented in our late effects and comprehensive care programme after paediatric hematopoietic stem cell transplantation (HSCT) for nonmalignant diseases. The programme focuses solely on screening of physical and mental health and on discussing PROs during the consultation. Methods The primary method of this study was semistructured interviews to explore the perspective of both patients and healthcare providers' (HCP) on the use of PROs, which were thematically analyzed. Additionally, an explorative quantitative approach with patient‐reported experience measures (PREMS) was used, with a pretest–posttest design, to assess whether the use of PROs was accompanied by more patient‐centred care. Results From the patient‐interviews (N = 15) four themes were extracted: use of PROs (1) help to discuss topics; (2) make the patients feel understood; (3) create a moment of self‐reflection; and (4) make consultations more efficient. Pre‐ and postimplementation analysis of PREMs (N = 40) did not show significant differences in terms of patient‐centeredness. Conclusion Our results demonstrate the added value of integrating PROs for health screening purposes within the long‐term follow‐up programme after paediatric HSCT, as perceived by both patient and HCP. With the active use of PROs, patients are stimulated to consciously assess their health status. Patient Contribution This study included patients as participants. Caregivers were approached if patients were below a certain age. Additionally, preliminary results were shared with all patients (including nonparticipants) during a patient conference day.


| INTRODUCTION
Patient-reported outcomes (PROs) are increasingly applied in hematopoietic stem cell transplantation (HSCT) for the purpose of collecting data for research or monitoring symptoms. 1,2However, PROs can also be used to better understand the patients' needs, and to support shared decision-making (SDM). 3,4Integrating PROs into routine care offers the healthcare provider (HCP) the opportunity to identify essential topics and address problems early on, provide personalized support, make timely referrals, and consequently improve quality of care. 5,6PROs have been incorporated into the late effects (LEEF) and comprehensive care follow-up programme after paediatric allogeneic HSCT for nonmalignant diseases.The integration of PROs in this programme was part of the implementation of value-based healthcare in this care path, aiming to enhance healthcare quality further.
HSCT has proven to be an intensive, curative treatment option for various severe paediatric diseases, including nonmalignant disorders such as inborn errors of immunity, hemoglobinopathies and bone marrow failure syndromes. 7,80][11][12][13][14] Proper screening for these late effects requires a dedicated long-term follow-up programme, which has been implemented at the Leiden University Medical Center (LUMC) in the Netherlands, providing comprehensive care from 2 years after HSCT onwards. 9,10The programme includes annual monitoring of both physical and mental health (Supporting Information S1: Figure 1), and continues throughout adulthood due to the potential for late effects to occur even many years after paediatric HSCT.
Current research on the value of PROs in healthcare has predominantly focused on diseases where intervention efficacy, symptom control, or cure were the primary treatment objectives. 6,15wever, the value of PROs has not been investigated in care paths for screening programmes, where active healthcare utilization and overt disease symptoms may be absent.Therefore, the aim of this study was to explore patients', caregivers' and HCPs' experiences with the active use of PROs during consultations in the late effects and comprehensive care (LEEF) programme after paediatric HSCT for nonmalignant diseases.Furthermore, the study aimed to evaluate the impact of PRO use on patient-centred care.

| PRO implementation
PROs were implemented in routine care in the LEEF programme in September 2021 (Figure 1).PRO domains from the International Consortium for Health Outcomes Measurement Standard Set 'Overall Paediatric Health' have been selected by consensus among both patients and the clinical team of the LEEF programme. 16Ageappropriate and validated patient-reported outcome measures (PROMs) were identified and selected based on their availability in Dutch (Supporting Information S1: Table 1).8][19][20][21][22][23][24][25][26][27][28] Patients completed PROMs before their consultation using the digital KLIK PROM portal (www.hetklikt.nu). 29In addition, patients completed a F I G U R E 1 Overview measurements over time.Shown are the measurements within this study over time.The measurements involve PREMs, PRO measures, and semistructured interviews.PREM, patient-reported experiences measures; PRO, patient-reported outcome.
symptom checklist (Supporting Information S1: Table 2).The HCP retrieved the PRO results in an electronic PROfile and discussed them with the patient during the consultation, for which the HCPs received training. 30

| Design
The primary method of this study was semistructured interviews to explore the perspective of both patients and HCPs on the use of PROs.Additionally, in anticipation of changes related to patientcenteredness, an explorative quantitative approach was used, with a pretest-posttest design, to assess whether the use of PROs was accompanied by more patient-centred care (Figures 1 and 2).

| Patient characteristics
Patient characteristics obtained from the medical files were age, gender and underlying disease (inborn errors of immunity, hemoglobinopathies and bone marrow failure disorders).

| Interviews with patients
Semistructured interviews were held from January to March 2022 to explore the patients' perspective on PRO use in all consecutive patients visiting the outpatient clinic.Participants were selected using convenience sampling.Interviews were held in-person or by video conference, depending on the participant's preference.Two researchers (F.Z. and N. G.) who were not involved in the patient's care, conducted the interviews.For participants below the age of 12, one-on-one interviews were conducted with their parents, while participants ages 12 and above had the choice of being interviewed individually, with their parents, or together.The initial interview topic guide created by the researchers (J.B., H. M. and A. h.P.) was revised after the first three interviews, as it was found to focus excessively on the questionnaires themselves (PROMs) rather than the use of PROs during the consultation (Supporting Information S1: Table 3).

| Interviews with HCPs
To gain a comprehensive understanding of the use of PROs within the LEEF programme, HCPs were interviewed as well.Currently, Flowchart showing inclusion of patients.Shown are the inclusion of patients in semistructured interviews and in the pretest-posttest design with PREMs.PREM, patient-reported experiences measures; PRO, patient-reported outcome.
three HCPs who use PROs work at the outpatient clinic.Two independent researchers (F.Z. and N. G.) took turns conducting the interviews with the HCPs.The topic guide created by researchers (J.B., F. Z. and N. G.) was adapted from the patients' topic guide (Supporting Information S1: Table 4).3) roles (two items). 34Both PREMs were translated into Dutch language level B1 by the Dutch Centre of Expertise on Health Disparities (Pharos institute), and were approved by an independent test panel (N = 3).Participants completed the PREMs in the digital KLIK PROM portal on two separate occasions: T 0 ) in before PRO implementation; T 1 ) within 7 days after the outpatient clinic consultation (Figure 1).If participants were below 16 years of age, their caregiver completed the PREMs.Inclusion for the PREM analysis closed 1 year after PRO implementation, ensuring that all patients who participated in the T 0 had the opportunity to participate in the T 1 measurement.

| Interviews
All interviews were recorded, transcribed verbatim and information was depersonalized.All interviews were conducted in Dutch.6][37][38] This guideline consists of a step-by-step method for analyzing qualitative data.Each researcher (J.B., F. Z. and N. G.) read and summarized the interviews, followed by the creation of conceptual interviews schemes.In these first steps, the relevant information from each interview is selected and clustered into different topics.After individual analysis, the researchers (J.B., F. Z. and N. G.) compared their findings, discussed their interpretation of the data, and reached consensus on a list of concepts (codes) linked to passages in each interview, enabling the identification of recurrent themes.Data collection and analysis took place simultaneously to enhance efficiency.Data collection continued until data saturation was reached, which was defined as no new upcoming themes in the analysis of the last three consecutive interviews.Finally, concepts were clustered into main themes and subthemes using the qualitative data analysis software ATLAS.ti(version 9). 39The HCPs' (N = 3) interviews were analyzed in the context of the themes identified from the patients' interviews.This approach was taken due to the small group size, which prevented data saturation from being achieved.

| PREMs
Statistical analysis of the PREM data was performed using SPSS version 25. 40 Mean scores of P3CEQ and PPPC-R at T 0 vs. T 1 were compared using paired sample t-tests.A p < .05 was considered statistically significant.The PREM analysis was performed after the completion of interview analysis, and the PREM results were not available to the HCPs or interviewers.Participants who were interviewed completed both PREMs before the interview to avoid influencing the PREM scores.

| Interviews with patients
In total, 15 of the 24 patients approached were interviewed after which data saturation was reached.Among the participants, eight out of 15 were male, ranging in age from 8 to 37 years (Table 1).The  7 and 8).

| Use of PROs help to discuss topics
Discussing the PROs to start the conversation Almost all participants briefly reviewed their PROs with their HCP, had discussed them and had decided together which topics needed more clarification.The participants were satisfied with the way the PROs had been discussed.Discussing the PROs helped the participants to gain clarity about the questions they had and helped to facilitate the discussion.Participants felt that through discussing the PROs, the HCP can extract topics that are essential for the patient more easily and address questions that arose from the PROs.This was perceived as valuable.Furthermore, participants reported that completing the PROMs prepared them to talk about sensitive or personal topics, such as mental wellbeing, instead of feeling overwhelmed when the HCP initiates these topics without prior notice.Lastly, a few participants mentioned that discussing the PROs served as a reminder to address specific issues with their HCP.

Impact of discussing PROs
The majority of participants reported that when discussing the PROs they felt safe to discuss any topic they desired, including sensitive or personal topics.However, a few participants considered certain aspects of the PROs to be too personal and therefore did not want to discuss them during the consultation.Additionally, two parents preferred to discuss topics without their children present in the consultation room.T A B L E 2 Illustrative quotations from Theme 1 'Use of PROs help to discuss topics'.HCP already had insights into their emotional state.The participants felt that the HCP was able to focus more on their needs.One participant expressed the need to provide context on PRO-related issues during the consultation to enhance the understanding of specific PROs.

Patient feels supported
Participants appreciated the time and attention of the HCP to evaluate the PROs.In addition, most participants experienced a sense of trust and support during their interactions with the HCP.Many participants appreciated that by evaluating the PROs, their well-being was actively monitored and were overall satisfied with the consultation.
Well, now we get the questionnaires that we have to fill in, so that's some sort of preparation.
T A B L E 5 Illustrative quotations from Theme 4 'Use of PROs make the consultation more efficient'.

Sex Age Quotation
♂ 8 I think it's beneficial that you don't have to discuss everything, then it's mostly the things you answered with yes or the things that are urgent, that are being highlighted and I think that is better, that saves time. [Caregiver] The other things, well the other answers I gave, were not different from the last time, so like, we didn't necessarily have to discuss these things.
mentally challenging and considered some questions as being too personal, such as the PROM regarding depressive symptoms.

| Use of PROs make the consultation more efficient
Some participants reported that using PROs made the consultation more efficient.The HCP was already aware of the most prominent issues, allowing irrelevant topics to be skipped or briefly touched upon, which was also preferred by half of the participants.A few participants noticed that their answers were directly transferred into their medical file, which meant the participants did not have to answer certain questions again.

| Additional results
Participants were specifically asked about the usefulness of PROs and opportunities for improvement of use of PROs.

Usefulness of PROs
While some participants did not personally perceive PROs as valuable because they did not have any problems to report, all participants emphasized the importance of PROs for those in need, healthcare improvements and research purposes.Some participants viewed the questionnaires within the context of active illness and treatment, and since they were already several years posttreatment, the PROs felt less relevant to them personally.However, participants could easily imagine that the PROs might be relevant to other patients.Most participants did not view completing the PROs as burdensome.Half of the participants regarded PROs as a valuable tool for monitoring the overall well-being and health of all patients.

Opportunities for improvement of use of PROs
Nearly all participants considered the implementation of PROs to be an improvement of care.PRO content was clear and appropriate for the consultation.However, the number of questions in the PROMs was excessive and some participants preferred to skip topics they considered irrelevant.Certain questions were sometimes perceived as too personal and detailed for an online questionnaire.One participant also noted that certain PROMs emphasized negative aspects too much and lacked a positive approach.Some participants thought that the recall period of the PROMs was inappropriate and wished for an extended timeframe.Lastly, a few participants found the completion of PROs challenging due to the language level of the PROs and requested support from caregivers.

| Interviews with HCPs
Two of the three HCPs worked with paediatric patients and one with adult patients.The median interview duration was 44 min (range: 39-47).All HCPs perceived that the use of PROs improve the consultations, improve insight into patients' overall well-being, and help to recognize and prepare topics needing attention during the consultation.The HCPs also noted that discussing the PROs led to more in-depth conversations and made it easier to discuss personal subjects, such as sexuality, as patients had already reflected on them and were not caught off-guard.However, the HCPs emphasized the need to verify patients' interpretation of the PROs and to conform or clarify any PRO-related issues.The HCPs noticed that patients were better prepared for consultations and were more involved in their care.This resulted in improved equality and reciprocity in the HCP-patient interactions, and improved SDM after PRO implementation.However, one HCP perceived a sense of detachment due to patients answering personal questions online instead of in person.All HCPs reported that consultations became more efficient due to improved preparation, although the time requires for consultation preparation had increased both for HCPs and patients.Nevertheless, the HCPs experienced that with PROs, patients and HCPs were better prepared, facilitating SDM.

| PREM
PRO implementation was evaluated by two PREMs (P3CEQ, PPPC-R) regarding patient-centeredness before (T 0 ) and after PRO implementation (T 1 ).T 1 measurement ended 1 year after PRO implementation and included 40 patients.Twenty-three patients were male.Age at T 1 ranged from 6 to 42 years (Table 6).Mean scores at T 0 and T 1 from the P3CEQ and PPPC-R were not significantly different (Table 7).
There was a trend (p = .09)for more attention to the factor 'context and relationship' (PPPC-R).2][43][44] According to a review from Carfora et al. 44 on the patient perspective regarding the use of PROs in clinical care, it is evident that various PROMs were utilized.The variations in PROMs used across studies can have implications for factors such as the length and number of questions in the PROMs, whether the PROMs were generic or disease-specific, the extent to which the PROs were discussed during consultations, and whether visual aids were used to aid in the interpretation of PRO results.
These variations could potentially influence patients' perceptions of the usefulness of PROs in their care.Nonetheless, there are many similarities between these studies and our results.
[46] Although SDM was not explicitly addressed by patients or HCPs, there are elements of SDM that were highlighted in the interviews.
Moreover, with the use of PROs, patients felt understood and supported by their HCP.HCPs reported a better understanding of their patient, enabling them to address personal topics more effectively.These factors could enhance the exploration of patient's values, thereby supporting SDM.Overall, research into the connection between SDM and the use of PROs in general has predominantly been conducted in a restricted range of care paths, rather than in care paths centred around health screening, such as the LEEF programme. 47Further research is needed to explore the association between the use of PROs and SDM.
Certain topics mentioned in the interviews also correspond to the quantitative analysis in this study.Although the evaluation using PREMs showed no significant difference in overall patientcenteredness after PRO implementation, detailed analysis did reveal a trend for more attention to the factor 'context and relationship', specifically related to the perceived compassion from HCPs and trust in HCPs.This factor also evaluates to which extent patients are comfortable discussing their problems with their HCP.These topics have been especially positively highlighted in the interviews, providing further support to the observed trend in the PREMs.
Unfortunately, due to the small sample size, possible significant differences could not be demonstrated.
PRO implementation impacted patients as well.Use of PROs was valuable for self-reflection and made patients feel more in control of their care, which is in line with previous research. 44,48However, the literature also suggests that self-reflection could be potentially stressful for patients. 41,44From our interview data, it remains uncertain whether self-reflection had resulted in stress.In the days prior and after the consultation prompted some patients to reflect on their time of hospitalization and the initial months following their discharge.This often led them to engage in discussions about this with their family.Still, it is not evident if this type of reflection induced stress.
Regarding possible effects of the use of PROs in the consultations, HCPs reported increased efficiency due to use of PROs, which has been described in the literature. 44,49However, in this study we did not measure the consultation duration nor the time devoted to certain topics.Further research is needed to assess the efficiency of the consultations when using PROs.There have been studies where PROs were utilized as a tool for evaluating active symptoms and determining the need for a health check at the hospital.However, providing context to PRO results was deemed essential by both patients and HCPs. 44,50Additionally, HCPs expressed the preference T A B L E 7 PREM: Mean PPPC-R and P3CEQ scores before (T 0 ) and after PRO implementation (T 1 ).for PROs to complement rather than replace regular consultations.
The HCPs highlighted that PROs should be used as an additional resource in the overall care process, providing valuable information to enhance patient-physician communication. 44,51is study has multiple strengths.First, this study is a multiple methods study.Most studies have either used quantitative measures or qualitative measures, such as focus groups.Second, the research team consisted of individuals from diverse professional backgrounds.This interdisciplinary collaboration brought different expertize and perspectives to the study, enhancing the identification of a wide range of themes and subthemes.However, there are also some limitations that should be acknowledged.First, this study did not conduct interviews before PRO implementation, precluding a comparison from pre-to postimplementation.Second, not all participants were able to review their PRO results since their consultation was conducted by phone.As a result, participants could not always determine whether the topics discussed were influenced by the PRO results.Third, in addition to PROMs patients completed a symptom checklist.Although the difference between PROMs and the symptom checklist was clear to HCPs, this may have been less clear to patients.The interviewers were aware of this issue and asked for clarification when necessary.Fourth, there was a lack of PREMs specifically aimed at evaluating PRO implementation.However, we expected differences regarding patientcenteredness based on literature and expert opinions, and therefore chose the PREMs accordingly. 3,44Fortunately, the themes derived from the interview data aligned with the PREM factors, providing additional support for the findings.Fifth, the sample size was too small to have sufficient power to assess changes regarding patientcenteredness.However, our results suggest the late effects and comprehensive care programme might already perceived to be focused on patient-centred care.This was evident when comparing the P3CEQ results to Dutch adults with a chronic condition, suggesting that the programme had already achieved a relatively high level of patient-centeredness. 33 It is possible that further improvement in patient-centeredness might be challenging to detect, as the programme was already performing well in this aspect.Lastly, there might be a potential bias in the study due to exclusion of patients who did not complete PROs, possibly underestimating the perceived usefulness and difficulty of the PROMs.
Topics addressed in the participants' interviews, such as language level, number of questions and personal inquiries, could be contributing factors.Reasons for not completing PROs should be further investigated.
Patients' inclusion criteria for both the interviews and the pretest-posttest study were: (1) allogeneic HSCT in childhood for a nonmalignant disease at the Willem-Alexander Children's Hospital with a follow-up of at least 2 years; (2) active follow-up at the LUMC outpatient clinic (LEEF programme); (3) completion of PROMs before the consultation; (4) Dutch-or English-speaking.Participants received complete study information and were recruited by telephone or in-person.This study was approved by the medical ethics committee of Leiden-The Hague-Delft (N20.181).Written informed consent was obtained from all participants.For participants aged 15 years or younger, additional assent was obtained from (both) caregivers.All HCPs (N = 3) involved in the LEEF programme were included.

2. 4 . 4 |
Patient-reported experience measures (PREMs) Two PREMs were used to assess if the use of PROs added value in terms of patient-centred care.PREMs were selected by the research team based on expert opinion.The Person-Centred Coordinated Care Experience Questionnaire (P3CEQ) consists of 11 items and is divided into two subscales: person-centeredness (eight items), and care coordination (five items). 31-33The Revised Patient Perception of Patient-Centeredness Questionnaire (PPPC-R) consists of 18 items with three factors: (1) healthcare process (eight items); (2) context and relationship (eight items) and ( median interview duration was 21 min (range: 11-46 min).Supporting Information S1: Table 6 shows details from nonparticipants (N = 9).Upon coding and categorizing the data, four main themes emerged: (1) use of PROs help to discuss topics; (2) evaluating the PROs make the patients feel understood; (3) completing the PROs create a moment of self-reflection; (4) use of PROs make the consultation more efficient.Additionally, participants were specifically asked about the usefulness of PROs and opportunities for improvement of use of PROs.Most participants completed the questionnaires (PROMs) independently, while some younger participants required assistance from their caregivers.Due to COVID-19 restrictions, participants over 18 years of age discussed the PROs with their doctor over the phone and were unable to view their results.Paediatric participants discussed their PROs during in-person consultations at the LUMC and had the possibility to review their results together with the HCP as they discussed it.Illustrative quotations are given per theme (Tables 2-5 and Supporting Information S1: Tables Abbreviation: HSCT, hematopoietic stem cell transplantation.

3. 1 . 3 |
Completing the PROs create a moment of self-reflectionCompleting the PROs prompted several participants to reflect on their current well-being, their transplantation experience, and everything they have been through since then.It also helped them to reveal issues for which they needed support.Many participants perceived the request to complete the PROMs as a way to prepare for the consultation, as it invited them to reflect on essential aspects of their lives.Parents found that the PROs served as a conversation starter with their children on topics such as alcohol and drug use.However, a few participants perceived completing the PROs as T A B L E 3 Illustrative quotations from Theme 2 'Evaluating the PROs make the patient feel understood'.

T A B L E 6
Abbreviations: HSCT, hematopoietic stem cell transplantation; PREM, patient-reported experience measure.a Time was calculated from measurement T 1 .

5 |
CONCLUSIONOverall, our results indicate that the use of PROs for screening purposes in the late effects and comprehensive care programme after paediatric HSCT is valuable from both patient's and HCP's perspectives.It is important to note that completing PROs should not replace routine consultations, as patients and HCPs have expressed the importance of providing context to the PRO results.The use of PROs can lead to more efficient consultations by addressing the essential topics identified through PRO analysis.Moreover, with the active use of PROs there might be a shift towards a more mutual patient-HCP relationship, in which patients are stimulated to consciously assess their health status.Future research could focus on linking PRO results to psychosocial and clinical outcomes, enabling further optimization of PROs as a screening tool and provide valuable insights into the relationship between PROs and patients' overall well-being.AUTHOR CONTRIBUTIONS Joëll E. Bense approached the participants, collected hematopoietic stem cell transplantation data and performed statistical analysis.F. Zwaginga and N. Guilonard conducted the interviews.Joëll E. Bense, F. Zwaginga, N. Guilonard, Hilda Mekelenkamp and Arwen H. Pieterse prepared the topic guide and analyzed the data.Marloes Louwerens, Anne M. Stiggelbout, Arjan C. Lankester and Anne P. J. de Pagter provided expert opinion.Joëll E. Bense wrote the manuscript.All authors critically revised the manuscript, approved the final version to be published and agreed to accountability of the work.
♀ 17 I thought it was better, because with the questionnaires you can really think clearly about everything beforehand and if you're at the appointment, well then you'll also forget half of it.♀ 30 … it's nice, because then I can address it if I have something on my mind.And most of the time, as I said before, she acts upon this and takes action, so that's really nice.[About discussing PROs] ♂ 11 I think it provides [name HCP] the right tools to start a conversation, so that you don't have to start asking questions out of the blue.[Caregiver about discussing PROs] ♀ 12 … and then its looked upon to try to understand how the patient is going through life, if she is supported, if she is happy or not happy, if there are potential gaps, if she is deeply unhappy, you name it.[Caregiver about discussing PROs] ♂ 37 It's a difficult subject.And by confronting people that are not necessarily burdened by these feelings, or well, burdened is maybe too strong, but simply don't have these feelings, well, it's not making it easier.[About PROs on depression] Abbreviations: HCP, healthcare provider; PRO, patient-reported outcome; ♀, female; ♂, male.BENSE ET AL. | 5 of 11 And she won't go through the questionnaire word for word, luckily, but I do notice that she asks substantive questions in such a way that I notice that she has read it, which I appreciate, because then I won't have done it for nothing and I can see that she is prepared.♀29WhenIfilled it out, about how I felt in the last week and everything, so what you're supposed to do.I filled it in, and then [HCP] also responded to it.[HCP]did ask like: what's the reason you filled it out like this.So, I did find that nice.
T A B L E 4 Illustrative quotations from Theme 3 'Completing the PROMs create a moment of self-reflection'.Sex Age Quotation ♂ 17 He completed the questionnaires two days beforehand and well, then you talk about it, you talk about the whole process and about how his friends dealt with it and well, you get to have a moment in which you talk extensively about it.[Caregiver] ♀ 17 I thought it was better, because with the help of the questionnaires you can really think clearly about everything.[About filling in the PROMs] ♀